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Abstract 

The rapid advancements in Extended Reality (XR) technolo- 
gies and their integration into the urban design process bring 
the definition of ‘space’ and its production to the forefront, 
where a new type of designable space that accompanies, 
surrounds, and complements physical urban interventions 
emerges: the virtual space. XR technology has opened up 
new avenues for creating virtual content in urban contexts. 
However, despite the potential for XR to fundamentally alter 
how we create and experience urban environments, there is 
a lack of literature on how to use XR to create immersive and 
accurately placed virtual experiences that complement our 
physical urban realities. 

This work aims to address this gap in the literature by 
exploring the potential of XR technology, specifically AR, 
for designing virtual urban space. The study follows a Re- 
search through Design approach (RtD), materializing findings 
through an experimental project in the form of a mixed-re- 
ality application that augments the city of Tampere, Finland. 
The developed application introduces virtual urban space as 
a space for cultural and artistic co-creation, offering the pub- 
lic a tool to reshape and envision their city through the lens of 
Augmented Reality (AR). 

This work offers insight into the latest tools and software 
development kits (SDK) used to augment large-scale urban 
areas. It proposes a solution for accurately placing virtual 
content in urban environments, and presents considerations 
for designing virtual urban space, as well as current technolo- 
gy limitations and future research avenues. 
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Introduction 
In today's world, ever-evolving technologies have seemingly 
swept the globe, blurring the line between what is real and 
what is not. This sensitive interplay between the physical and 
the imaginary has touched upon many aspects of our every- 
day life and has, in some cases, formed a new framework for 
understanding the world around us; creating novel ways to 
experience ordinary things. 

Recent technologies have made it possible for different 
realities to exist and merge with the one we live in and inter- 


act with. The term Extended Reality (XR), an umbrella term 
that comprises Virtual Reality (VR), Augmented Reality (AR), 
and Mixed Reality (MR), but is not limited to these three, has 
come into existence and people are slowly escaping every- 
day life into this newly emerging form of reality that can be 
designed, adapted, and harnessed to improve the way we ex- 
perience our surroundings (Liu et al., 2017). 

XR technology has opened up new avenues for the cre- 
ation of virtual content in urban contexts. However, despite 
the potential for XR technology to fundamentally alter how 
we create and experience urban environments, there is a lack 
of literature on how to use XR to create immersive and accu- 
rately placed virtual experiences that complement our phys- 
ical urban realities. 

This work aims to address this gap in the literature by ex- 
ploring the potential of XR technology, specifically AR, for de- 
signing virtual urban space. The paper begins by reviewing the 
existing literature on XR technology and its use in urban de- 
sign and envisioning future urban scenarios. It then presents 
a case study in Tampere, Finland that showcases the poten- 
tial of using XR in designing virtual urban experiences while 
presenting the latest tools that facilitate this process. Finally, 
the paper discusses the opportunities XR technology offers 
and provides suggestions for further research in this area. 


Background 


The use of AR for Urban Design: 

Definitions and Justification 

AR systems are those that allow real and virtual objects and 
information to coexist in the same space and provide re- 
al-time interaction. They allow the user to view the real world 
with overlaid virtual objects and enrich the user's experience 
and perception of reality with additional information. Howev- 
er, AR does not replace the real world as in the case of Virtual 
Reality (VR) (Redondo et al., 2017). According to Klopfer and 
Squire (2008), AR could be defined as “a situation in which a 
real-world context is dynamically overlaid with coherent lo- 
cation or context-sensitive virtual information” (p. 205). It is 
the synthesis of computer images in the real world (Zachary 
et al, 1997). And it can be identified by three basic features: 1) 
It combines real and virtual environments, 2) It is interactive 
in real-time, and 3) It provides accurate three-dimensional 
registration of real and virtual objects (Azuma,1997). Mobile 
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Augmented Reality (MAR) provides the aforementioned fea- 
tures without constraining the individual's location to a spe- 
cially equipped area. It can be used anywhere, allowing the 
user to access AR experiences and add layers of information 
to any environment anytime through mobile devices (H6llerer 
and Feiner, 2004). 

AR and VR share some similarities in terms of immersion, 
navigation, and interaction (Redondo et al., 2017). Unlike VR, 
which completely immerses the user in a virtual environment, 
AR allows users to experience the real world through a virtual 
overlay (Bower et al., 2014). It is important to highlight that the 
main goal of a VR system is using technology to replace rath- 
er than complement reality and create a new immersive envi- 
ronment, while the main goal of an AR system is to enhance 
reality by overlaying it with virtual content in a complimentary 
non-immersive way that enriches the user's perception of the 
real environment (Billinghurst et al, 2015). 

In a discipline such as Urban Planning and Design, in which 
spatial and contextual information are core components of 
professional practice, Squire and Klopfer (2007) suggest that 
AR applications hold a particular promise and provide enor- 
mous potential for enhancing the design process due to their 
ability to connect academic content and practices with our 
physical, lived worlds. Potts et al. (2017) mention a shift and a 
direction toward the integration of AR in planning and design- 
ing public spaces in the physical world, and emphasize the 
possibility that urban planners will need to engage in both, the 
physical and the virtual placemaking process in the future. 


Augmented Reality in Urban Design 

The introduction of Augmented Reality applications into the 
architecture and urban design study and practice has provid- 
ed a new channel of interaction with the built environment 
and produced a need for a new set of skills and a higher level 
of digital proficiency alongside traditional paper-based design 
and analysis skills to be developed in order to utilize these 
tools to their full potential (Indraprastha and Shinozaki, 2009). 

Billinghurst et al. (2015) expand on the significance of aug- 
mented reality (AR) in the fields of architecture, urban plan- 
ning, and urban design by pointing out that while 2D plans can 
show a building's layout in great detail, it would be challenging 
for the client to visualize the building through 2D drawings. 
Physical models, 3D renderings, and interactive flythroughs 
can therefore provide a solution to this problem, but they are 
scaled-down representations of the final soace. These might 
not accurately portray the building's final image, including its 
final scale and placement within the urban context. The use of 
AR allows users to view full-sized 3D virtual models of future 
buildings placed in their intended contexts. (Calabrese and Ba- 
resi, 2017; Redondo et al, 2017). This allows AR to be utilized as 
a planning tool, placing different variations of the same build- 
ing on-site and allowing for stakeholders’ feedback, collabora- 
tion, and participation in the design process. 

In addition to serving as a tool for visualizing future projects 
and urban development scenarios, Mesárošová et al. (2015) 
present an example of an application used in Urban Plan- 
ning education called “Visionary Cities” created by AR Group 
Manusamo&Bzika. Through AR, the application provides the 
possibility to view and explore cities and concepts that were 
never built and remained as sketches, such as projects done 
by Archigram (A neo-futuristic avant-garde architectural 
group formed in the 1960s (Sadler, 2005)). 


Envisioning Future Urban Scenarios Using XR 

In today's world, there is a need for new urban visions that 
address pressing urban issues, concerns, and transforma- 
tions such as overcrowding, social inequality, environmental 
degradation, climate change, urban gentrification, increased 
refugee influx, increased surveillance, technology and the 
smart city discourse in urban planning, etc. (Sevilla-Buitra- 
go, 2013; Leorke, 2020). Augmented, virtual, and mixed re- 
ality can be powerful and effective tools that support the 
conceptualization, visualization, and communication of im- 
agined urban futures. 

Astudy by Shawash and Marji (2020) explores the potential 
of Mobile Augmented Reality (MAR) technologies as a tool for 
public engagement with ecologically sustainable urban regen- 
eration projects. They present two urban development scenar- 
ios for envisioning a greener version of the city of Amman, Jor- 
dan by virtually reintroducing the city’s enclosed and dried-up 
water stream and proposing sustainable water management 
solutions through an interactive table-top AR model of the site. 
The study envisions “what if” scenarios of the city’s only water 
stream, and uses AR to present this vision to the public, engag- 
ing them in the dialogue of imagining a greener, more sustain- 
able future for their city. 

A study by Thibault et al. (2021) proposes a fictional city, 
one that utilizes augmented city technologies "to produce cre- 
ative and playful spaces for citizen—urban interventions” and 
places the people in the center of the place-making process. 
The proposed model uses the city as a resource for activities 
and initiatives that allow the people to shape their environment 
and envision shareable alternate scenarios, strengthening their 
right to be represented and to use the city. Residents can ex- 
plore, create, and modify the city's virtual augmentations, and 
develop a sense of ownership over its public spaces (Thibault 
et al, 2021). The authors suggest that the required technolo- 
gies that allow us to interact in unprecedented ways with the 
urban environment seem to be available and what is needed is 
to combine them in the right way. 


The Challenges of World-scale AR 

Calabrese and Baresi (2017) explore placing urban design 
proposals and student projects in an outdoor urban context 
in the city of Milan. They attempt to enhance the accuracy 
of placing virtual design interventions by detecting specific 
3D models and using beacons (small, wireless transmitters 
that use Bluetooth technology to send signals to other near- 
by devices). However, the proposed solution adds additional 
complexity to the average user and requires the detected 3D 
object to be always in view. 

Rohil and Ashok (2022) explore the use of building infor- 
mation modeling (BIM) software along with AR visualization to 
present two types of urban planning scenarios: the creation of 
new structures and the recreation of existing designs. They of- 
fer a more recent overview of the workflow and possible tools 
that can be used to integrate AR into architecture and urban 
design. Nevertheless, the presented results showcase a pro- 
posed design floating in space rather than being placed in its 
designated location and in the correct orientation and scale, 
and little is discussed on the potential of utilizing visual posi- 
tioning systems (VPS) for enhancing placement accuracy and 
designing compelling virtual experiences. 

Although many studies attempted to explore and use AR 
in the context of urban design and to augment urban space, 
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very few address the accurate placement of virtual content in 
urban areas. Whether the AR layer presented proposed build- 
ings, urban development proposals, image/video overlays, cre- 
ative 3D models, etc, its placement in the real world has faced 
challenges due to GPS and localization inaccuracy (Blum et al., 
2013; Calabrese and Baresi, 2017; Redondo et al., 2017; Rohil 
and Ashok, 2022). The introduction of AR elements in urban 
contexts, whether marker-based AR (detecting a certain image 
or QR code through a device's camera to place virtual content) 
or marker-less AR (using a device's camera to detect vertical 
and/or horizontal planes or 3D objects in the surrounding envi- 
ronment to place virtual elements) or even location-based AR 
(utilizing specific location coordinates to load virtual content 
in specific areas), the placed virtual content in a large outdoor 
setting would shift, jitter, disappear, get misplaced, and move 
with the user’s movement as they attempt to explore it. 

Moreover, manual placement of large-scale virtual ele- 
ments using a device's input (e.g, touch screen), such as build- 
ings, urban furniture covering an entire street, indoor virtual 
content, etc. is inefficient and poses additional challenges and 
limitations to the users, especially if they are new to the tech- 
nology and are not aware of the designer's decisions regarding 
optimal placement. 

Another important consideration when designing XR expe- 
riences is the user’s point of view (POV), which holds substan- 
tial implications for the design, perception, and overall efficacy 
of the immersive environment. 

Utilizing a first-person perspective, or world-scale AR, ena- 
bles users to perceive and engage with the urban environment 
from a human-scaled perspective, resulting in a more immer- 
sive experience. In urban XR experiences, the first-person per- 
spective facilitates a more profound understanding of spatial 
relationships, architectural scale, and the experiential qualities 
of the built environment. This can offer insightful information 
on how people will use the built environment and how it will 
affect their perception of the area. This POV, however, can be 
constrained in terms of the quantity of data that can be con- 
currently displayed on the screen, which can make it harder to 
assess the design as a whole. 

Alternatively, users are able to observe the complete area 
from a bird's eye view when using the third person perspec- 
tive, such as in Tabletop AR, which offers a more objective and 
thorough view of the urban environment. the third-person per- 
spective positions the user as an external observer, detached 
from the immediate context, and often overlooking the virtu- 
al environment from an elevated or distanced vantage point. 
This POV provides users with an overarching view of the urban 
fabric, enabling a more comprehensive understanding of the 
spatial structure, urban patterns, and connectivity. However, it 
may be less realistic and not as effective when expressing how 
people interact with the built environment compared to the 
first person POV and may reduce the sense of immersion and 
personal agency within the XR environment. 

It's critical to consider which POV to use depending on the 
nature of the project and the desired feedback. For instance, 
first person POV might be more effective for assessing the pe- 
destrian experience of a brand-new public space, while third 
person POV could be used for assessing the effects of a major 
development on the local urban fabric. The POV can also af- 
fect the usability and accessibility of XR technology for various 
users, the level of depth and accuracy necessary in designing 
the experiences, and the techniques used for evaluation and 


feedback. 

The case study discussed in the following sections pre- 
sents an example of utilizing XR technologies in co-creating 
virtual urban space. It explains the used tools and solutions 
for creating accurately placed, immersive virtual content in 
the city. 


Methodology 

This study draws on a comprehensive review of literature 
in the fields of XR technology, urban design and planning, 
as well as Human-Computer Interaction (HCI) and follows 
a Research through Design approach (RtD); “an approach to 
conducting scholarly research that employs the methods, 
practices, and processes of design practice with the inten- 
tion of generating new knowledge” (Zimmerman and Forlizzi, 
2014, p. 167). This work materializes insights and knowledge 
through an experimental project in the form of a mixed-re- 
ality application that augments the city of Tampere, Finland. 
The developed application introduces virtual urban space as 
a space for cultural and artistic co-creation, offering the pub- 
lic a tool to reshape and envision their city through the lens 
of augmented and mixed reality. To investigate the use of XR 
technology in designing virtual urban space, the presented 
case study involved observing and documenting the design 
process, the performance of the mixed reality application 
prototype, as well as users' reactions to it. 

The ideation and development of the application prototype 
were done in the following steps: 

1) Literature search on the use of XR in urban contexts as 
well as current challenges and technology limitations. 

2) Identification of the latest open-source and available 
tools for augmenting city-scale spaces. 

3) Schematic analysis of Tampere city's layout and opti- 
mal locations for introducing virtual content. 

4) Informal interviews with local Finnish artists regarding 
the need for exhibiting their art in public spaces as vir- 
tual content. 

5) Collection and design of context-specific virtual con- 
tent to be placed in selected locations in the city. 

6) Development and testing of the mixed reality applica- 
tion prototype using MAR and AR glasses. 

This work aims to offer insight into the latest technologies 
used to augment large-scale urban areas, considerations for 
designing virtual urban space, as well as current limitations 
and future research avenues. 


Case Study: the Tampere xRT Project 

Tampere, Finland's third-largest city, is regarded as a hub for 
business, technology, and culture. The city is renowned for its 
cutting-edge technological research and advancements as 
well as its vibrant cultural scene, which features a wide range 
of museums, theaters, and festivals. 

This project offers individuals access to alternative and 
novel viewpoints of the city that can enhance their daily lives 
by leveraging Tampere's diverse cultural experiences and the 
potential of XR. 

By allowing users to shape their own virtual urban surround- 
ings, the Tampere xRT project examines the idea of placemak- 
ing through XR and presents the city as an open exhibition. It 
offers a sustainable and accessible platform where local and 
international artists, cultural venues, and the public can share 
their visions of the city, by showcasing their art in virtual urban 
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space. The project explores how the public realm of Tampere 
can be co-created with XR through the collaboration of artists 
and the public. 


Problem and proposed solution 

For providing immersive and accurate AR experiences in out- 
door urban settings, the project aimed to tackle the AR place- 
ment-accuracy challenges and offer an experience that can 
run on mobile devices as well as AR glasses. Different loca- 
tions around the city were selected, those included a neigh- 
borhood park (Figure 1), a pedestrian bridge in the city center, 
and a landmark in Tampere's city center to explore the effec- 
tiveness of the AR solution in a 1) green setting, 2) an active 
pedestrian route, and 3) a distinguishable building and plaza. 
The design of virtual content in urban space not only provides 
a playful and interactive experience but can also enrich the 
public's perception of the city. This offers opportunities for 
visualizing future urban developments, historical informa- 
tion, and even fictional and fantastical scenarios as a form 
of speculative design that can widen users’ imagination and 
allow them to rethink their surroundings. 


Figure 1. Left: user viewing virtual content in a park using AR glasses, 
Right: the user's view through the AR glasses showing virtual 3D model in the park 


Developing Urban XR Experiences: 
Tools and Technicalities 
The project utilizes Visual Positioning Systems (VPS) to iden- 
tify the users’ location and accurately place the virtual con- 
tent in physical space. The process begins with mapping 
specific locations in the city by using photographs, creating 
a spatial map and point clouds of the location, and extracting 
feature points that can be then detected, ensuring accurate 
localization (finding the position and orientation) and place- 
ment of the virtual content. For this process, the “Immersal” 
software development kit (SDK) was used, and the created 
maps of the scanned locations were then inserted into “Uni- 
ty”, a game development engine, for designing and determin- 
ing the location of the virtual content that will augment the 
physical space (Figure 2). 

The Immersal SDK facilitates the process of creating loca- 
tion-aware spatial environments using any mobile device, of- 


Figure 2. Left: Unity's work space showing the point clouds of the bridge and the 
placement of virtual content, Right: mobile screenshot of the bridge area through the 
AR app, showing the placed virtual content in its physical location. 


fering fast visual positioning with low system overhead. It en- 
ables devices to precisely locate and orient the users’ view in 
the physical world, allowing accurate placement of persistent 
virtual content in their surroundings. 

In addition to using the Immersal SDK, Google's Geospatial 
API recently released in 2022 was also used to add 8D virtu- 
al elements in locations that were not priorly scanned using 
Immersal’s tools. Google's Geospatial API allows developers 
to use Google's pre-scanned data and feature points used in 
Google Street View AR to accurately place virtual content in 
any place in the world. However, the capabilities offered by 
Immersal’s mapping tools by scanning specific building fa- 
cades offer element-specific augmentation. This means that 
virtual content can be placed on certain windows, walls, orna- 
mentation, roofs, etc. 

Nevertheless, the localization of the devices faced some 
challenges after sunset, since the spatial maps were created 
from images captured during the daytime. This was resolved 
by mapping the environment at different times of the day to 
capture the context in different lighting conditions and en- 
sure correct AR placement and environment detection in all 
settings. 

The projects’ results were shared with the public, as well as 
Tampere's municipality and multiple cultural venues including 
the Tampere theatre, Tampere Art Museum, and Tampere City 
Library, which all presented a positive attitude toward the intro- 
duction of XR in the city and the proposed platform. 


Conclusion 

XR technology is rapidly developing and the intersection be- 
tween the real and the virtual is increasing at a quick pace. 
This study discussed the use of XR in urban contexts, the de- 
sign of virtual urban space, and the tools available today to 
facilitate this process. Following an RtD approach, this work 
presented a case study that explores the latest software de- 
velopment kits for creating accurately placed AR content in 
large urban settings. Moreover, the study offered a possible 
solution for a challenge frequently faced while exploring the 
usability of AR in outdoor contexts. 

XR offers great potential for the fields of architecture, urban 
planning, and design, and allows us to envision and experience 
future urban scenarios in real-time on-site and in full scale. 

For architects, urban designers, planners, and other pro- 
fessionals involved in shaping the built environment, XR tech- 
nology opens up new design options, promotes teamwork, 
streamlines decision-making, and boosts productivity as it can 
be connected with other technologies like Building Informa- 
tion Modeling (BIM), Geographic Information System (GIS), and 
Internet of Things (IoT). Additionally, by focusing on more im- 
mersive and interactive experiences rather than 2D drawings 
and models, XR has the potential to revolutionize the design 
discipline, resulting in new processes, tools, and techniques for 
design as well as new ways for practitioners and stakeholders 
to collaborate and communicate with one another. 

However, the current state of XR technology, specifically 
MAR is limited by the utilized hardware and requires large pro- 
cessing power, which can result in low refresh rates, device 
overheating, and a lack of immersion for users with older de- 
vices. Moreover, further research is needed on designing good 
quality and context-specific virtual content for augmenting 
urban spaces without overpopulating them and creating visual 
noise for the users. Several social and regulatory challenges 
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are also presented when implementing XR technologies in the 
urban realm, those include but are not limited to accessibili- 
ty, safety, privacy, impact on the built environment, as well as 
legal and regulatory issues, which could be explored in future 
research. 
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